Chinese Grammatical Error Correction Using Pre-trained Models and Pseudo Data

In recent studies, pre-trained models and pseudo data have been key factors in improving the performance of English grammatical error correction (GEC) task. However, few studies examined role Chinese GEC Therefore, we develop based on three models: BERT, T5, BART, then incorporate these with to determine best configuration for On natural language processing computing (NLPCC) 2018 shared task test set, all our single outperform ensemble developed by top team BART achieves an F score 37.15, which is a state-of-the-art result. We combine kinds data: Lang-8 (MaskGEC), Wiki (Backtranslation). find that most can benefit from data, BART+Lang-8 (MaskGEC) ideal setting terms accuracy training efficiency. The experimental results demonstrate effectiveness provide easily reproducible adaptable baseline future works. Finally, annotate types development data; show word-level errors dominate types, word selection must be addressed even when using data. Our codes are available at https://github.com/wang136906578/BERT-encoder-ChineseGEC .